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Overview 



Over the past few decades, a postsecondary credential has become increasingly important in the labor 
market, and college attendance has become more common. Unfortunately, however, many students leave 
college before receiving a degree, particularly those who are academically underprepared for college- 
level work. Many postsecondary institutions operate learning communities to promote students’ involve- 
ment and persistence in college. Learning communities typically place groups of students in two or more 
linked courses with mutually reinforcing themes and assignments. They seek to build peer relationships, 
intensify connections to faculty, and deepen understanding of coursework. While learning communities 
are increasingly popular, little rigorous evidence on their effects exists. 

As part of MDRC’s multisite Opening Doors demonstration, Kingsborough Community College in 
Brooklyn, New York — a large, urban college with a diverse student population that includes many im- 
migrants — operated one such learning community program. The program placed freshmen in groups of 
up to 25 who took three classes together during their first semester: an English class, usually at the devel- 
opmental level; an academic course, such as health or psychology; and a one-credit orientation course. 
The program provided enhanced counseling and tutoring as well as a voucher for textbooks. 

Using a rigorous research design, MDRC assigned 1,534 freshmen, at random, either to a program group 
that was eligible for the learning community or to a control group that received the college’s standard 
courses and services. Analyses in this report show that: 

• The program improved students’ college experience. Students in the program group felt more 
integrated and more engaged than students in the control group. 

• The program also improved some educational outcomes while students were in the learning 
community program, but the effects diminished in subsequent semesters. Program group stu- 
dents, for example, attempted and passed more courses and earned more credits during their first se- 
mester. 

• The program moved students more quickly through developmental English requirements. Stu- 
dents in the program group were more likely to take and pass English skills assessment tests that are 
required for graduation or transfer. 

• The evidence is mixed about whether the program increased persistence. Initially the program did 
not change the rate at which students reenrolled. In the last semester of the report’s two-year follow-up 
period, however, slightly more program group members than control group members attended college. 

These results are not the final word on Kingsborough’ s program: MDRC plans to continue tracking sam- 
ple members’ outcomes for at least another year. 
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Preface 



Community colleges are “the Ellis Island of American higher education,” according to 
the January 2008 report of the National Commission on Community Colleges. Serving nearly 
12 million students annually, they provide a pathway into the middle class for many low- 
income individuals, including people of color, immigrants, full- and part-time workers, and stu- 
dents who are the first in their families to attend college. However, the increased access to post- 
secondary education that community colleges offer has not always translated into individual 
success for students. As many as 60 percent of incoming students at community colleges re- 
quire at least one developmental (or remedial) course, and too many drop out before getting a 
credential — often because they never get beyond developmental classes. This report offers 
promising evidence that a one-semester “learning community” intervention can provide an early 
boost to freshmen, helping students move more quickly through developmental English re- 
quirements and earn more credits in their first semester. 

At Kingsborough Community College in Brooklyn, freshmen were placed in groups of 
up to 25 who took three classes together during their first semester: an English class, usually at 
the developmental level; an academic course, such as health or psychology; and a one-credit 
orientation course. The program also provided enhanced counseling and tutoring and a textbook 
voucher. Kingsborough’ s program is part of MDRC’s Opening Doors demonstration, which is 
testing interventions at six community colleges designed to help low-income students stay in 
school and succeed. Opening Doors is the first large-scale study of community college pro- 
grams to use a random assignment design, the “gold standard” of program evaluation. 

The report concludes that the jury is still out on whether learning communities improve 
students’ persistence in school. Although educational outcomes improved for students while they 
were in the learning community program, these effects diminished in later semesters, as the con- 
trol group students caught up. However, at the end of the follow-up period — three semesters after 
the intervention — there was an indication that Opening Doors students were somewhat more 
likely to be enrolled in college. As a result of these promising findings, Kingsborough has ex- 
panded its learning communities program, with the goal of serving 80 percent of incoming fresh- 
men by 2010. Currently, the program reaches about 65 percent of incoming freshmen. 

MDRC plans to continue following the students at Kingsborough to find out whether 
the learning community intervention has lasting effects. In addition, with our partners at the Na- 
tional Center for Postsecondary Research, we have launched a multi-college demonstration of 
learning communities that will test variations of this promising model. 



Gordon Berlin 
President 
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Executive Summary 



Over the past few decades, a postsecondary credential has become increasingly impor- 
tant in the labor market, and college attendance has become more common. Unfortunately, 
however, many students leave college before receiving a degree, particularly those who are aca- 
demically underprepared for college-level work. Many postsecondary institutions operate learn- 
ing communities to promote students’ involvement and persistence in college. Learning com- 
munities typically place groups of students in two or more linked courses with mutually rein- 
forcing themes and assignments. They seek to build peer relationships, intensify connections to 
faculty, and deepen understanding of coursework. While learning communities are increas- 
ingly popular, little rigorous evidence on their effects exists. 

As part of the Opening Doors demonstration and evaluation project jointly under- 
taken by MDRC and the MacArthur Foundation- funded Network on Transitions to Adult- 
hood, six participating colleges operated innovative programs aimed at increasing students’ 
achievement and persistence and, eventually, their graduation rates and earnings. Kingsbor- 
ough Community College in Brooklyn, New York — a large, urban college in the City 
University of New York (CUNY) system — tested a program called Opening Doors Learn- 
ing Communities. The program placed freshmen, most of whom failed one or more of the 
skills assessment tests that all incoming students take, into groups of up to 25 who took 
three classes together during their first semester. It also provided enhanced counseling and 
tutoring as well as a voucher for textbooks. 

This report discusses the program’s implementation and its effects on students up to 
two years after they entered the study. Using a rigorous research design, MDRC randomly 
assigned students either to a program group that was eligible for the learning community or 
to a control group that received standard college courses and services. Any subsequent sub- 
stantial differences in outcomes can be attributed to the Opening Doors program. 

In summary, the key findings from this report are: 

• The program improved students’ college experience. Students in the pro- 
gram group felt more integrated and more engaged than students in the con- 
trol group. 

• The program improved some educational outcomes while students were 
in the learning community, but the effects diminished later. Program 
group students passed more courses and earned more credits during their first 
semester. 
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• The program moved students more quickly through developmental 
(remedial) English requirements. Students in the program group were 
more likely to take and pass the college’s English skills assessment tests that 
are required for graduation or transfer. 

• The evidence is mixed about whether the program increased persistence 
in college. Initially the program did not change the rate at which students re- 
enrolled. At the end of the report’s follow-up period, however, slightly more 
program group members than control group members attended college. 



How Was the Program Implemented? 

Opening Doors Learning Communities — operated between 2003 and 2005 — 
placed groups of freshmen into three li nk ed classes: an English course, usually at the devel- 
opmental level; an academic course, such as health or psychology; and a one-credit orienta- 
tion course. The instructor of the orientation course provided enhanced counseling to stu- 
dents, and the program provided enhanced tutoring as well as a voucher for textbooks. 

The program was targeted to freshmen who planned to attend college full time dur- 
ing the day and who did not test into English as a Second Language. ESL students were ex- 
cluded because they participated in another learning community program. For the same rea- 
son, students in four “career majors” were excluded for the first year of the study. Over 
three-fourths of the students were under age 21 when they entered the study. Reflecting the 
diversity of the student body at Kingsborough, 38 percent identified themselves as black, 27 
percent as white, and 20 percent as Hispanic. Almost three-fourths of the students in the 
study reported that they or at least one of their parents were born outside the United States. 

The following key findings on the implementation of Kingsborough’s learning 
communities program are based on interviews with and surveys of the college’s administra- 
tors, faculty, staff, and students. 

• The program’s key features were all in place when operations began, 
and they remained in place throughout the study. 

Despite a compressed planning period, Kingsborough’s program was well imple- 
mented from the start. This achievement reflects the college administration’s strong com- 
mitment to the program and the study. 

• The learning communities varied in class size and the degree to which 
faculty integrated their courses. 



ES-2 



Over four semesters, Kingsborough ran 40 learning communities for the study: 3 1 
with developmental English and 9 with college-level English. Owing to several challenges 
— including the difficulty of predicting how many students would test into each English 
level — class size varied from 6 to 25 students, with an average of 17. 

All instructors developed a new syllabus or revised their regular syllabus for the 
learning community, and all learning communities had some joint assignments across 
classes. The degree of integration across the courses varied, however, as did the frequency 
of joint assignments. The instructors in most learning communities met regularly to discuss 
student progress and coordinate assignments, but, in a minority of communities, the instruc- 
tors rarely met. Thus, the study provides a strong test of the structural features of the learn- 
ing community, but it may not fully detect the effects of tightly integrating course curricula. 



How Was the Impact of the Program Evaluated? 

As noted above, to determine the effect, or “impact,” of Kingsborough’s program, 
MDRC assigned students, at random, to either a program group or a control group. Random 
assignment occurred just before students registered for classes. The study is tracking both 
groups over time to determine whether the learning community program results in better 
outcomes for students. Random assignment ensures that the motivation levels and personal 
characteristics of students in the two groups were similar when the study began; hence, any 
subsequent differences in outcomes can be attributed to the program. The study is estimat- 
ing the value added of Opening Doors, above and beyond what students normally receive. 
Kingsborough offers a rich array of academic programs and services, so the bar is set rela- 
tively high for the program to surpass. Also, the study examines whether the package of 
reforms and enhancements in Opening Doors at Kingsborough led to different outcomes, 
compared with standard classes and services. The study will not, however, disentangle the 
effects of each component. 



Did the Program Make a Difference? 

This report discusses the program’s impacts on a range of educational outcomes. 
The learning communities program directly affected students during their first semester at 
Kingsborough. Many higher education experts believe that students’ academic and social 
experiences during that first semester play a substantial role in their future success — that 
students who develop strong initial connections with the material they study, with other stu- 
dents, and with faculty are more likely to persist in college than students who do not. Also, 
at Kingsborough, students who make better progress in meeting their developmental Eng- 
lish requirements may be more motivated to stay in school. 
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This report presents impacts for the full research sample at Kingsborough (1,534 stu- 
dents) for up to two years after students entered the study. The key impact findings follow. 

• The program improved students’ experiences in college. 

When surveyed approximately a year after entering the study, students in the pro- 
gram group reported that they felt more integrated at school and were more engaged with 
their coursework, instructors, and fellow students and had a stronger sense of belonging 
than did control group students. They were more likely to say that their courses required 
critical thinking and that they had acquired valuable academic and work skills. Finally, they 
were more likely to rate their college experience as “good” or “excellent.” These findings 
strongly suggest that the learning community program provided a markedly different ex- 
perience for students. These results are similar to findings from some past studies of learn- 
ing communities. 

• The program improved several educational outcomes for students dur- 
ing the semester that they participated in the learning community. 

MDRC examined students’ academic performance by using transcript data from 
Kingsborough. Newly enrolling students were randomly assigned for the study just before 
the start of their first semester in college (enrollment occurred in four different but contigu- 
ous semesters: fall 2003, spring 2004, fall 2004, and spring 2005). The first semester that 
each student was in the study is called the “program semester.” 

Figure ES.l illustrates some key outcomes during the program semester. The solid 
bars show the average outcomes for program group members, and the white bars show the 
averages for control group members. The difference between each pair of bars represents 
the program impact, if any, and asterisks indicate whether an impact is “statistically signifi- 
cant,” meaning that it is unlikely to be due to chance. (All impacts discussed in the Execu- 
tive Summary are statistically significant.) As the two sets of bars at the left of the figure 
show, students in the program group attempted and passed about half a course more at 
Kingsborough during their first semester in the study than control group students did. They 
also earned almost one more “developmental credit” (called an “equated credit” at Kings- 
borough). Developmental courses do not earn college credit, but they do count in determin- 
ing whether a student is attending school full time. Program group members were also more 
likely to pass all their courses during the first semester (not shown). 

• The positive effects on educational outcomes diminished in later semes- 
ters of the two-year follow-up period. 

The program generated a small increase in the number of credits attempted and 
earned in the first postprogram semester, but the effects dissipated later. By the end of the 



ESA 



The Opening Doors Demonstration 
Figure ES.l 

Key Impacts During the Program Semester 
Kingsborough Community College Report 
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Courses attempted Courses passed 




Regular credits earned Developmental credits 

earned 



Impact 

SOURCE: MDRC calculations from Kingsborough Community College transcript data. 



NOTE: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 



two-year follow-up period, program group members earned an average of two and a half credits 
more at Kingsborough than control group members. (This gain is primarily due to impacts dur- 
ing the program semester and, to some extent, during the first postprogram semester.) 

• Opening Doors Learning Communities helped students move more 
quickly through the college’s developmental English requirements. 

A goal of Kingsborough’s program was to help students more quickly complete de- 
velopmental requirements and progress to college-level English. To enroll in the college- 
level course at Kingsborough, students must first pass the CUNY reading and writing skills 
assessment tests. (Students must pass the reading, writing, and math skills assessment tests 
in order to transfer to a four-year CUNY institution.) Students take the skills 
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The Opening Doors Demonstration 
Figure ES.2 

Impacts on English Skills Assessment Tests 
Kingsborough Community College Report 
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NOTES: Outcomes include data from the program semester through the second postprogram semester. 
Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 



tests prior to enrolling; if they do not pass the reading and writing tests, they typically can 
retake them only after they pass specific developmental English courses. 

Figure ES.2 shows the proportion of the two research groups who took the tests dur- 
ing their first three semesters in the study and passed the tests by the end of that period (in- 
cluding students who passed the tests before starting their freshman year). The program in- 
creased the proportion of students who attempted and passed the tests. Although not illus- 
trated in the figure, most of these impacts are driven by effects in the first (program) semes- 
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ter. It is notable, however, that the control group members had not “caught up” in terms of 
test taking and passing by the end of the follow-up period. 

MDRC also examined progression through English courses for different subgroups 
of the research sample. Among the subset of the sample who failed both English skills as- 
sessment tests before starting their freshman year, program group members were more like- 
ly than their control group counterparts to enroll in developmental English during their first 
two semesters. Program group members who failed one of the tests before entering college 
were also more likely to enroll in developmental English during their first semester and 
were more likely to enroll in and pass college-level English during their first two semesters. 
The program did not affect progression through English courses among students who had 
passed both English assessment tests before starting their freshman year. 

• So far, the evidence is mixed about whether Kingsborough’s program 
increases student persistence in college. 

A central goal of all the Opening Doors programs is to increase persistence in col- 
lege. Initially, Kingsborough’s program did not change the rate at which students re- 
enrolled. In the last semester of the follow-up period, however, a difference emerged: 53 
percent of the program group registered for at least one course that semester at Kingsbor- 
ough, compared with 48 percent of the control group. Data from the National Student Clear- 
inghouse, which provides enrollment information at most colleges in the nation, shows a 
similar effect on persistence emerging that semester. MDRC plans to continue tracking out- 
comes to see if this effect remains. 



What Are the Implications of the Results? 

Opening Doors Learning Communities at Kingsborough substantially improved 
students’ experiences in college and some key educational outcomes while they were in the 
program, but, for the most part, the effects did not persist. MDRC plans to track sample 
members’ outcomes for at least three years after their random assignment to the study to 
determine the longer-term effects on their academic performance, persistence, and gradua- 
tion as well as on their later employment rates and earnings. Thus, the results in this report 
are not the last word on Kingsborough’s program. That said, the findings point to the fol- 
lowing conclusions. 

• Kingsborough’s learning community model shows promise as a strategy 
to help students move through developmental education. 

Many students begin at community colleges academically underprepared for col- 
lege-level work. Research shows that approximately 60 percent of freshmen beginning at 



ES-7 



community college need at least one developmental-level course . 1 Students with very low 
skill levels can spend a year or more in developmental courses, and many leave school be- 
fore completing developmental requirements. A key challenge is how to help students meet 
these requirements so they can eventually complete college. 

This report’s findings suggest that Kingsborough’s Opening Doors Learning Com- 
munities model is one strategy that college administrators could consider. Students in the 
program group were more likely to pass the English skills assessment tests — the gateway 
to college-level English. Also, program group students who failed one of the tests before 
entering school were more likely to take and pass college-level English during the follow- 
up period. It is important to highlight that Kingsborough’s program included English in the 
learning community. As a result, students in the program group were required to take Eng- 
lish, and, as discussed above, the program substantially increased the proportion of students 
who took developmental English. Because students must pass developmental classes before 
retaking the assessment tests, this program feature is central to the impact on test taking and 
passing. 



• The results from Kingsborough suggest that the jury is still out on 
whether learning communities improve students’ persistence. 

As noted, Kingsborough’s program did not increase students’ retention in college 
until the third postprogram semester, and MDRC will collect more follow-up data to deter- 
mine whether the effect continues. At this juncture, however, it is worthwhile to consider 
the absence, so far, of a strong program effect on retention. 

Kingsborough’s program was based on the hypothesis that a more engaging and 
successful first semester would lead to more successful future semesters and higher rates of 
retention. One may wonder, however, how much change in college attendance is reasonable 
to expect from a one-semester program. Individuals make life choices, including whether or 
not to remain in college, based on myriad factors — many outside the college environment. 

Even if Kingsborough’s program does not lead to substantial retention effects, it 
could still generate increases in degree receipt, transfer, and other longer-term outcomes, 
since the program group students who were still enrolled at the end of the report’s follow- 
up period are somewhat further along in school than the control group members. 

• Enhanced services that last longer than one semester may have a more 
substantial effect on students. 



'Clifford Adelman, Principal Indicators of Student Academic Histories in Postsecondary Education, 
1972-2000 (Washington, DC: U.S. Department of Education, 2004). 
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Kingsborough’s program lasted one semester. The college’s administrators decided 
that there was no practical way to maintain the linked-course structure after the first semes- 
ter, since students needed and wanted to take a variety of different courses in subsequent 
semesters. Also, the program was designed on the assumption that students’ early experi- 
ences at college influence their later success, and administrators believed that students 
should transition into the regular college community as quickly as possible. 

The question of how long a learning community program should continue is compli- 
cated. That said, the results from the Kingsborough study suggest that participating in a learning 
community program for more than one semester may yield more substantial effects — the posi- 
tive effects on academic outcomes were by far the largest during the first semester. If the op- 
tions of a multiple-semester learning community or participating in a different learning commu- 
nity after the first semester are not possible, colleges could offer other kinds of enhanced ser- 
vices in later semesters, such as intensive counseling or more financial support. 

It is worth noting that, in some of the other sites in the Opening Doors demonstra- 
tion, the early results follow a similar pattern: Effects are largest when students receive en- 
hanced services, and they diminish or even disappear after the services end. 

* * * 

The study at Kingsborough is of a specific program model, targeted to a certain 
group of students, in a particular setting. Other learning community models, target groups, 
and institutional settings may well lead to different results. Another rigorous study, the 
Learning Communities demonstration, was launched in 2006 and is using random assign- 
ment to test the effects of learning communities in up to six colleges or universities. The 
demonstration is part of the National Center for Postsecondary Research, funded by the 
U.S. Department of Education. 
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Chapter 1 

Introduction 



This report presents results from a rigorous study of a learning communities pro- 
gram for incoming freshmen at Kingsborough Community College in Brooklyn, New York. 
Operated from 2003 to 2005 as part of the multisite Opening Doors demonstration, Kings- 
borough’ s learning communities program placed groups of up to 25 students each in three 
linked classes. The program also provided enhanced counseling and tutoring and a voucher 
to pay for textbooks. This report presents information on how the Opening Doors program 
was implemented at Kingsborough and what its effect are, up to two years after students 
entered the study. This is the first in a series of reports that will present detailed findings 
from all the Opening Doors demonstration sites. 

This report updates and supplements the early results from the Kingsborough study 
that were presented in a report in 2005. 1 The 2005 report, which examined the first quarter 
of the sample members to enter the study, showed that the Opening Doors program led to 
better academic outcomes during students’ first semester at Kingsborough, including pass- 
ing more courses and increased completion of developmental English requirements a year 
after students entered the study. 

This chapter begins with an overview of the Opening Doors demonstration. It then 
focuses specifically on the study at Kingsborough, describing the college and its Opening 
Doors program model. The chapter concludes with a brief description of the contents of the 
rest of this report. 



Overview of the Opening Doors Demonstration and Evaluation 

This section discusses the importance of community colleges as a focus of study 
and provides some information about the development of the Opening Doors demonstra- 
tion. It also describes the study’s research design, guiding conceptual model, and key re- 
search questions. 

Why Focus on Community Colleges? 

Community colleges make higher education affordable and accessible to virtually 
anyone seeking the opportunity. Today, about 1,200 community colleges serve nearly 12 



'Bloom and Sommo (2005). 
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million students. Almost half of all college students nationwide attend a community col- 
lege. 2 Compared with four-year institutions, community colleges enroll more students of 
color and more low-income students. They are also more likely to enroll working adults and 
parents. 3 

Community colleges prepare students for transfer to four-year colleges and univer- 
sities, and they provide training in a wide variety of occupations. As shown in Figure 1.1, 
recent data from the U.S. Bureau of the Census indicate that, in 2005, an adult with an asso- 
ciate’s degree earned an average annual income of about $38,000, compared with only 
about $29,000 for an adult with a high school diploma. 4 Given the widening earnings gap 
between individuals with a postsecondary credential and individuals with a high school di- 
ploma, community colleges represent a potential pathway out of poverty and into the mid- 
dle class. 

Unfortunately, although many people attend community colleges, only a minority of 
students end up receiving a degree. The U.S. Department of Education reported that only 
about one-third of students who enter community college with the intention of earning a 
degree accomplish this goal at any institution of higher education within a six-year period. 5 
Completion rates are particularly low for students who begin college academically under- 
prepared and must take developmental-level courses. Recognizing the importance of higher 
education today, college administrators, policymakers, and researchers are searching for 
effective strategies to help community college students stay in school and succeed. 

The Opening Doors Programs 

Building on previous efforts to leam about factors that affect community college stu- 
dents’ enrollment and completion, MDRC, with support from a consortium of funders, launched 
the Opening Doors demonstration in 2003 . A review of prior research and focus groups with 
past, current, and potential community college students 6 revealed some key factors that hinder 
students’ progress: underpreparation for college-level work; the challenges of juggling school, 
work, and family; and institutional barriers, such as inadequate support services and insufficient 
financial aid. Opening Doors is testing three promising strategies that colleges could adopt to 
address these factors and increase student achievement and persistence: 



2 American Association of Community Colleges Web site. 

’Horn and Nevill (2006). 

4 The data shown in the figure are correlational; they do not indicate that a certain credential causes a 
certain level of earnings. 

5 U.S. Department of Education (2002). 

6 See Matus-Grossman and Gooden (2002) for a discussion of the focus group results. 
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The Opening Doors Demonstration 
Figure 1.1 

Average Annual Earnings, by Educational Attainment: Adults, Nationwide, 2005 
Kingsborough Community College Report 



Bachelor's degree 



Associate's degree 



High school diploma 



Without a high school 
diploma 




SOURCE: U.S. Bureau of the Census (2007). 
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• Curricular and instructional innovations, including learning communities 
in which students take blocks of classes with the same group of peers, cus- 
tomized instructional support, academic instruction for students on academic 
probation, and enhanced orientation courses to help students navigate the col- 
lege experience 

• Enhanced student services, including stronger, more personalized academic 
advisement, career counseling, and tutoring 

• Supplementary financial aid, such as special scholarships or money di- 
rected to specific education-related costs, like vouchers for textbooks 
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After a nationwide search, MDRC selected six community colleges in four states to 
participate in the demonstration and evaluation. MDRC worked with each college to de- 
velop an Opening Doors program that combined two or three of the strategies listed above. 
Table 1.1 lists the participating colleges and provides a brief summary of their Opening 
Doors programs and the groups of students who were targeted and served. Findings from 
the evaluation yield information about how to improve the rate of student success and will 
speak to the focus of the Spellings Commission — the U.S. Secretary of Education’s Com- 
mission on the Future of Higher Education — on the need for investments that lead to better 
student outcomes. 

As noted above, this report focuses on Kingsborough Community College, which, 
in the fall of 2003, was the first demonstration site to begin operating its Opening Doors 
program. A later section of this chapter provides some information on Kingsborough and its 
Opening Doors program. 

The Research Design and Conceptual Model 

The evaluation of the Opening Doors programs is being conducted by MDRC, a 
group of scholars who are part of the Mac Arthur Foundation- funded Research Network on 
Transitions to Adulthood, and an expert on the relationship between education and health at 
Princeton University. 7 To measure the effects of Kingsborough’s program, as well as the 
effects of the other colleges’ programs, the evaluation is using a random assignment re- 
search design, a first in large-scale community college research. At each college, students 
who met the site’s eligibility criteria and agreed to take part in the study were assigned, at 
random, either to a program group that received enhanced Opening Doors services or to a 
control group that received the college’s standard services. The study is tracking both 
groups over time to find out whether the enhanced services result in better outcomes for 
students. Random assignment ensures that the motivation levels and personal characteristics 
of students in the program group and control group were similar at the beginning of the 
study; hence, any subsequent substantial differences in educational or other outcomes can 
be attributed with a high level of confidence to Opening Doors. The study is estimating the 



7 Members of the Research Network on Transitions to Adulthood are Gordon L. Berlin (MDRC), 
Mark Courtney (University of Washington), Sheldon Danziger (University of Michigan), Connie A. 
Flanagan (Pennsylvania State University), Frank F. Furstenberg (University of Pennsylvania), Vonnie C. 
McLoyd (University of North Carolina, Chapel Flill), Wayne Osgood (Pennsylvania State University), 
Jean E. Rhodes (University of Massachusetts, Boston), Cecilia E. Rouse (Princeton University), Ruben G. 
Rumbaut (University of California, Irvine), Richard Settersten (Oregon State University), and Mary C. 
Waters (Harvard University). Christina Paxson of Princeton University is leading the evaluation compo- 
nent focused on health outcomes. 
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The Opening Doors Demonstration 



Table 1.1 



Opening Doors Programs and Target Groups 



Kingsborough Community College Report 



Site Brief Program Description 

Chaffey College College Survival Skills and Enhanced 

Rancho Cucamonga, California Student Services: Students took a one- 

semester guidance course that provided 
instructional support as well as advising; 
students were required to visit the col- 
lege’s Success Centers, which provided 
extra academic support. 



Target Group 

Students ages 18-34 on aca- 
demic probation who earned 
fewer than 35 credits and who 
either had a cumulative grade 
point average below 2.0 (C) or 
did not complete at least half the 
courses in which they enrolled. 



Delgado Community College 
and Louisiana Technical Col- 
lege-West Jefferson 

New Orleans area, Louisiana 



Kingsborough Community 
College 

Brooklyn, New York 



Lorain County Community 

College 

Elyria, Ohio 



Owens Community College 

Toledo, Ohio 



A Scholarship Predicated on Academic 
Performance: Students were eligible for 
$1,000 scholarship for each of two semes- 
ters; scholarship was tied to maintaining 
at least half-time enrollment and a grade 
point average of 2.0 (C). 

Learning Communities and a Book 
Voucher: Groups of students took three 
linked courses together; students received 
enhanced advising and tutoring and 
vouchers to pay for textbooks. 

Enhanced Student Services and a Mod- 
est Scholarship: Students were assigned 
to an Opening Doors adviser with a small 
caseload with whom they were expected 
to meet frequently; students had access to 
designated contact in financial aid office; 
students were eligible for $150 scholar- 
ship for each of two semesters, paid after 
mandatory meetings with adviser. 

Enhanced Student Services and a Mod- 
est Scholarship: Students were assigned 
to an Opening Doors adviser with a small 
caseload with whom they were expected 
to meet frequently; students had access to 
designated contact in financial aid office; 
students were eligible for free one-on-one 
tutoring and for $150 scholarship for each 
of two semesters, paid after mandatory 
meetings with adviser. 



Parents ages 18-34 whose family 
income was below 200 percent 
of the federal poverty level. 



Incoming freshmen ages 17-34 
who planned to attend college 
full time. 



Students ages 18-34 whose fam- 
ily income was below 250 per- 
cent of the federal poverty level 
and who either were incoming 
freshmen or had completed 
fewer than 13 credits and had a 
history of academic difficulties. 



Students ages 18-34 whose fam- 
ily income was below 250 per- 
cent of the federal poverty level 
and who either were incoming 
freshmen or had completed 
fewer than 13 credits and had a 
history of academic difficulties. 



SOURCE: MDRC field research data. 



5 



value added of Opening Doors, above and beyond what students at the colleges normally 
receive. 



In designing the evaluation and developing the key research questions, MDRC and 
its research partners considered the basic conceptual model illustrated in Figure 1.2. The 
model provides a framework for linking the Opening Doors reforms to various outcomes 
that are important for a successful educational experience and transition to a better life. As 
illustrated in the figure, the Opening Doors reforms in curriculum and instruction, student 
services, and financial aid theoretically affect some key “early” educational or academic 
outcomes, while the program is operating (such as the number of credits completed and 
academic performance), as well as some “later” educational outcomes, including semester- 
to-semester persistence, graduation, and transfer to another postsecondary institution. 

As Figure 1.2 illustrates, the effects on the educational outcomes, if they are posi- 
tive and strong, would in turn lead to improved labor market outcomes in the longer term, 
including better jobs and higher earnings and enhanced well-being. Some of the social, per- 
sonal, and health outcomes (such as social networks or sense of self) might be affected in 
the short term, while others (such as physical and mental health) are more likely to be af- 
fected only in the longer term. 

The model illustrated in Figure 1.2 shows some basic relationships and, as noted 
above, helped guide the design of the study. As Opening Doors evolved, it became clear 
that some of the relationships observed in the study may be a bit more complicated. For ex- 
ample, in a learning communities program, such as the one tested at Kingsborough, groups 
of students take classes together as a cohort. This, in itself, could lead to changes in social 
networks, connections with peers, and similar outcomes. In fact, as discussed below, social 
changes such as these are central to the theory behind why learning communities might help 
students succeed. The relationships, in other words, are not all linear and one-directional, as 
illustrated in the simplified figure. This report examines a range of educational and well- 
being outcomes. Chapters 4 and 5 provide more information on the hypotheses about the 
likely effects of Kingsborough’ s program. 

Key Research Questions 

The Opening Doors evaluation includes three main components: an implementation 
analysis, an impact analysis, and a cost analysis. The key research questions in the imple- 
mentation analysis follow: 

• Were the enhanced services sufficiently distinct from the services available 
to the study’s control group to constitute a “fair test” of the intervention? 
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The Opening Doors Demonstration 
Figure 1.2 

Basic Conceptual Model for Evaluating the Effects of Opening Doors Programs 
Kingsborough Community College Report 
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• What services were provided as part of the Opening Doors programs? What 
was their quality and intensity, and how might the programs help students? 

• How were the programs managed and operated, and how were the services 
delivered? 

Chapter 3 of this report addresses these implementation questions for Kingsborough’s 
Opening Doors program. 

As discussed above, the random assignment design provides a rigorous way to esti- 
mate the effects of the enhanced programs on various outcomes of interest. Reflecting the 
outcomes shown in Figure 1.2, the key research questions in the analysis of the programs’ 
effects, or impacts, follow: 

1 . Do the Opening Doors enhancements in curriculum, student services, and fi- 
nancial aid in community colleges lead to more positive early educational 
outcomes — including completing more credits and earning better grades — 
compared with standard college courses and services? (The study is measur- 
ing the early educational outcomes only at the Opening Doors colleges, not 
other postsecondary institutions. See Chapter 2 for more detail on the data 
sources used in this report and what they measure.) 

2. Do the enhancements lead to more positive later educational outcomes, in- 
cluding higher rates of persistence in school, of degree attainment, and of 
transfer to four-year institutions? 

3. Do the enhancements or the resulting positive educational effects have a 
positive impact on students’ personal development, social networks, civic 
participation, and health behaviors? 

4. Do the enhancements or the resulting positive educational effects impact stu- 
dents’ success in the labor market? 

Chapter 4 provides some answers from the study at Kingsborough to the first two 
questions, and Chapter 5 provides some answers to the third. The impacts presented in this 
report cover up to two years following random assignment. MDRC will continue to track 
outcomes for the research sample for at least three years to understand the longer-term ef- 
fects of the Kingsborough program, including on labor market outcomes (the fourth ques- 
tion listed above), and will present the results in a future report. 

In each site in the study, MDRC and its research partners collected a range of data 
to help answer the impact and implementation research questions, including student tran- 



script data and a survey that was administered about a year after students entered the study. 
Chapter 2 provides more information on the data sources used in the analyses in this report. 
In addition, MDRC has collected data on the costs of operating each of the Opening Doors 
programs, as well as on the costs of providing each college’s standard classes and services. 
A future report will discuss the costs of the programs. 

A Limitation of the Research Design 

A random assignment evaluation is considered the most reliable way to test pro- 
grams such as those in Opening Doors, but this study’s design has an important limitation. 
The focus groups with students and the other research that MDRC conducted when design- 
ing Opening Doors pointed to the need for programs with multiple enhancements and sup- 
ports. Also, it was important to ensure that the program interventions were as robust as pos- 
sible and clearly different from the colleges’ standard offerings. As a result, as noted above, 
each college operated a program with at least two different components. The study will de- 
termine whether that package of enhancements led to different outcomes, compared with 
the standard college classes and services. The evaluation will not, however, disentangle the 
effects of each component. 

In Kingsborough, the study will determine whether the package of the learning 
community, enhanced counseling and tutoring, and textbook voucher lead to different stu- 
dent outcomes, compared with unlinked, standard courses and the college’s standard ser- 
vices. It will not, however, determine the specific effect of each of the program’s compo- 
nents, such as the enhanced counseling or textbook voucher. In addition, the study will not 
determine the effect of the difference in class size discussed in Chapter 3 or in any variation 
in quality of instruction. The implementation research at each college will shed light on 
which program dimensions may matter the most, but it will not yield definitive answers. 
That question will need to be addressed in future studies. 



Kingsborough Community College and Its Opening Doors 
Program 

This section describes Kingsborough and its setting and the origins of the college’s 
Opening Doors Learning Communities program. It also briefly describes the program 
model. The section begins by providing some background on learning communities, the 
central element of Kingsborough’s program. 
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Some Background on Learning Communities 

Learning communities are seen by many as a promising strategy to promote student 
involvement with faculty, peers, and the subject matter they are studying, as well as student 
success. 8 Many experts believe that learning communities are particularly promising for 
students taking developmental (remedial) courses. 9 A key monograph on learning commu- 
nities offers the following definition: “any one of a variety of curricular structures that link 
together several existing courses — or actually restructure the material entirely — so that 
students have opportunities for deeper understanding and integration of the material they 
are learning, and more interaction with one another and their teachers as fellow participants 
in the learning enterprise.” 10 The four most common models of learning communities are 
paired or clustered courses, cohorts in large classes, team-taught programs, and residence- 
based programs. 11 The first model — upon which Kingsborough’s program is based — 
li nk s two or more individually taught courses. From 20 to 30 students take the courses to- 
gether as a cohort, and, often, faculty redesign the curricula for the separate courses. Also, 
the courses are block scheduled, so that they meet Monday through Thursday, usually be- 
tween 9 A.M. and 2 P.M. 

The roots of the learning community approach can be traced back as far as the 
1930s, to an experimental program at the University of Wisconsin that redesigned the first 
two years of college around an interdisciplinary study of democracy in ancient Athens and 
modern-day America. 12 The learning community models that are more familiar today, such 
as those described above, began to spring up in the 1970s. By 2002, the National Survey of 
First-Year Academic Practices found that 62 percent of responding colleges enrolled at least 
some cohorts of students into two or more courses. However, at most colleges, these pro- 
grams involved only a small proportion of students. For example, about 60 percent of two- 
year colleges enrolled at least some students in learning communities, but fewer than 20 
percent of these colleges enrolled more than 10 percent of freshmen in such programs. 13 

Many community colleges adopt learning communities with the goal of improving 
the retention, persistence, and success of their most vulnerable students. Tinto and Eng- 
strom have conducted and synthesized research on student attrition and articulated the theo- 
retical underpinnings of learning communities. 14 They posit that increasing students’ at- 

8 The section on learning communities is adapted from Bloom and Sommo (2005); Price (2005); and 
Richburg-Hayes, Visher, and Bloom (2008). 

9 See, for example, Grubb (2001). 

10 Gabelnick, MacGregor, Matthews, and Smith (1990). 

1 'Levine Laufgraben and Shapiro (2004). 

12 Shapiro and Levine (1999). 

“Barefoot (2002). 

14 Tinto (1993, 1997, 1998); Engstrom and Tinto (2007). 
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tachment to the college and increasing their engagement through academic and social ex- 
periences in college are important factors in improving persistence and retention. According 
to Tinto, building a sense of community, both academically and socially, is central to solv- 
ing attrition problems. 15 Astin studied both individual and organizational factors that were 
associated with persistence and found that engagement, particularly formed by interactions 
among students and between faculty and students, was important. 16 

An excellent resource that summarizes much of the research on learning communi- 
ties is from the National Learning Communities Project monograph series. 17 Many studies 
of learning communities have discussed the programs’ implementation and described stu- 
dents’ and instructors’ experiences. Some evidence suggests that learning communities can 
increase students’ integration and sense of belonging in the college community and their 
overall satisfaction with their college experience. For example, one study cited in the 
monograph found that learning communities helped students make connections across dis- 
ciplines and with peers. 18 Another recent, large-scale study of learning communities for 
academically underprepared students in 19 colleges found statistically significant impacts 
on engagement, the quality of relationships with classmates and faculty, and differences in 
classroom activities, such as participation in class discussions and using ideas from other 
courses. 19 

Despite the breadth of research on the learning community experience, few studies 
have measured the effect of learning communities on key student outcomes, such as persis- 
tence, course completion, and degree attainment. A few studies cited in the monograph 
mentioned above found that students in learning communities had the same grades as or 
better grades than similar students who were not in learning communities and were more 
likely to remain in college. 20 The study of learning communities for academically underpre- 
pared students in 19 colleges (see the preceding paragraph) found a modest but statistically 
significant positive effect on retention one year after enrolling in a learning community, 
averaged across the colleges. 21 

In sum, although past research on learning communities has found generally prom- 
ising results, few studies have employed rigorous research designs, and none of the large- 
scale studies have used a random assignment research design. The Opening Doors random 



15 Tinto (1993). 

16 Astin (1993). 

17 Taylor (2003). 

18 Russo (1995); cited in Taylor, Moore, MacGregor, and Lindblad (2003). 

19 Engstrom and Tinto (2007). 

2(l Tinto (1997); Tinto, Russo, and Kadel (1994); Gordon, Young, and Kalianov (2001). 
21 Engstrom and Tinto (2007). 
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assignment evaluation at Kingsborough provides an important contribution to the growing 
body of research on learning communities. 

Kingsborough Community College and Its Environment 

Before describing Kingsborough’ s Opening Doors Learning Communities program, 
it is worthwhile to consider the college and its environment. Kingsborough Community 
College is the only community college in Brooklyn, the largest borough of New York City, 
with a population of roughly 2.5 million people. Brooklyn has no majority racial/ethnic 
group: 43 percent are white; 35 percent are black; 9 percent are Asian; and 20 percent are 
Hispanic (any race). Close to half of all Brooklyn residents speak a language other than 
English at home. In 2006, 23 percent of Brooklynites lived below the federal poverty 
level. 22 This poverty rate is higher than the rate across the five boroughs of New York City, 
19 percent, and the national poverty rate, 13 percent. In 2006, the average unemployment 
rate in Brooklyn was 7.4 percent, a bit lower than the rate across the five boroughs of New 
York, 7.8 percent. The unemployment rate in the United States that year was 6.4 percent. 23 

Founded in 1963, Kingsborough is one of six community colleges in the City Uni- 
versity of New York (CUNY), the nation’s largest urban university system. 24 A large urban 
college, Kingsborough’ s 70-acre campus boasts an unexpectedly scenic location next to a 
quiet residential neighborhood at the southern tip of Brooklyn, on the Atlantic Ocean. The 
campus has numerous outdoor sculptures and an array of amenities, including a beach, an 
aquarium, and an art gallery. The college has more than 800 staff and an operating budget 
of nearly $60 million. 

Kingsborough serves about 35,000 students annually. The college reports that its 
students come from 1 10 countries and speak 68 languages. Kingsborough’s student body is 
younger than that of many other community colleges — nearly three-fourths of Kingsbor- 
ough students are under age 25 — and a large fraction of students (about half) attend full 
time. The college’s Web site reports that Kingsborough ranks in the top 3 percent of com- 
munity colleges nationwide in the number of degrees awarded to minority students. 25 



22 The federal poverty level in 2006 for a family of four was $20,614 (U.S. Bureau of the Census, 
2006b). 

"The statistics in this paragraph are from the 2006 American Community Survey (U.S. Bureau of the 
Census, 2006a). 

24 CUNY also includes 11 four-year institutions, a graduate school, a law school, and a school of 
biomedical education. 

2 TCingsborough Community College Web site: www.kbcc.cuny.edu. 
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Community colleges in New York State are among the most expensive in the na- 
tion: 26 Full-time Kingsborough students pay more than $3,000 per year in tuition and fees. 
At the same time, New York has an unusually generous state-funded financial aid program, 
the Tuition Assistance Program (TAP), which provided more than $800 million in grants to 
nearly 400,000 students in 2003. Nearly 70 percent of Kingsborough students receive some 
form of federal or state aid. 

The college, which accepts all students with a high school diploma or General Edu- 
cational Development (GED) certificate, provides developmental-level courses and English 
as a Second Language (ESL) courses to help prepare students for other college courses. 
Kingsborough offers a wide array of associate’s degree programs, including career-oriented 
programs and traditional liberal arts programs. Kingsborough’ s Office of Continuing Edu- 
cation operates the college’s certificate training programs, customized training for business, 
preparation for the GED certificate exam, and other noncredit programs, while the academic 
side of the college operates its associate’s degree programs. 

The distinction between “credit” and “noncredit” programs at Kingsborough is 
complicated by the fact that many students in the degree programs take noncredit develop- 
mental English and math courses because they fail one or more of the skills assessment tests 
that are administered to all students who enter the CUNY system. Developmental courses at 
Kingsborough do not earn any college credit, although each such course is assigned 
“equated credits” to account for the hours spent in class. Equated credits count in determin- 
ing whether a student is attending school full time. 

In 1998, the CUNY Board of Trustees, with the strong support of Mayor Rudolph 
Giuliani, voted to eliminate developmental (remedial) classes from the university’s four- 
year colleges. Once this policy took effect, students who failed any of the three skills as- 
sessment tests that are administered prior to enrollment — which covered reading, writing, 
and math — were directed to a community college for developmental coursework. The 
change generated great controversy. Critics, citing data showing that students who begin at 
community colleges are less likely to complete bachelor’s degrees than comparable students 
who begin at four-year colleges, argued that the change would ultimately reduce the number 
of poor and minority students who obtained bachelor’s degrees. 27 

Although all incoming freshmen at Kingsborough have earned a high school di- 
ploma or GED, only a small minority pass all three skills assessment tests. In 2003, when 
the Opening Doors study started, only 1 8 percent of incoming students passed the reading, 



26 American Association of Community Colleges (2003). 
27 See, for example, Lavin and Weininger (1999). 
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writing, and math tests — a rate that is typical for many urban community colleges. Stu- 
dents who fail the tests are not required to take developmental courses when they start 
school, but students in all of Kingsborough’s associate’s degree programs must complete 
any required developmental English courses, plus two credit-bearing English courses that 
are open only to students who have passed both the reading and the writing skills tests. 
Similarly, students cannot transfer to a CUNY four-year college until they have passed all 
three of the skills tests. In general, students who fail the skills tests before enrollment can 
retake the tests only after they have passed specific developmental courses. 

In sum, the skills assessment tests at CUNY act as a “gatekeeper” to an associate’s 
degree or transfer to a four-year institution. In addition, as noted above, graduation rates for 
students who begin college in need of developmental education are particularly low. A key 
challenge and focus at Kingsborough, and in higher education more broadly, is how to ef- 
fectively help students complete developmental requirements and move to college-level 
courses. (As discussed below, most of the students in the Opening Doors study at Kings- 
borough took a developmental English course. A goal of the program was to help students 
move more quickly through developmental English requirements, allowing them to take 
college-level English.) 

Origins of the Opening Doors Learning Communities Program 

Kingsborough’s first learning communities program began in the mid-1990s and 
targeted English as a Second Language (ESL) students who were entering degree programs. 
Later, the college created another learning community program that targeted students in 
four “career majors”: accounting, business, mental health, and early childhood education. 
Data collected by the college showed that students in the learning communities had higher 
rates of course completion and semester-to-semester retention and higher grade point aver- 
ages than students at Kingsborough who were not in a learning community. 

Based on this positive experience, the leadership at Kingsborough was eager to ex- 
pand learning communities to a broader group of students. MDRC approached the college’s 
leadership in 2002 to discuss the possibility of Kingsborough’s participation in the Opening 
Doors demonstration, which offered an opportunity not only to expand learning communi- 
ties but also to rigorously test their effects on students. With special funding from the Robin 
Hood Foundation, which supports programs that aim to reduce poverty in New York City, 
Kingsborough developed the Opening Doors Learning Communities program and started 
operating it in fall 2003. 
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The Opening Doors Learning Communities Model 

Kingsborough targeted its Opening Doors Learning Communities program to stu- 
dents who met the following criteria: 

• Was a first-time incoming freshman who planned to attend college full time 
during the day 

• Did not test into ESL (that is, tested into either developmental English or col- 
lege-level English) 

• Was age 17 to 34 28 

ESL students were excluded from the study because they participated in the special 
learning communities program mentioned above. Similarly, students in the four “career ma- 
jors” for which a separate learning community operated were excluded for the first year of 
the study. The career learning community program ended after the 2003-2004 academic 
year, and, after that, students in those majors were included in Opening Doors. 

In designing Opening Doors Learning Communities, Kingsborough administrators 
were especially interested in targeting liberal arts majors, because they believed that many 
students in that group do not have clear academic or career goals and so might benefit from 
a model that provided enhanced structure and support. In addition, they made an effort to 
target students who missed the CUNY systemwide application deadline and applied directly 
to Kingsborough, often just weeks or even days before the start of classes. College data 
showed that these two overlapping groups of students tended to have poor outcomes, sug- 
gesting that they might benefit from Opening Doors. 

The program placed students in groups of up to 25 that took three classes together 
during their first semester at Kingsborough, forming a learning community. The paired or 
clustered courses consisted of an English course, usually at the developmental level; an 
academic course required for the student’s major; and a one-credit freshman orientation 
course. In addition to the linked courses, the Opening Doors program also offered addi- 
tional components designed to help students succeed: enhanced counseling and support, 
which was provided by the instructor of the orientation course; enhanced tutoring; and text- 
book vouchers for use at the college’s bookstore. Chapter 3 provides more detail on Kings- 



28 During the first semester of program operations, Kingsborough’ s Opening Doors program was open 
only to students between ages 18 and 34 who reported household income below 250 percent of the federal 
poverty level. In subsequent semesters, the income criterion was removed, having been deemed unneces- 
sary because such a large proportion of Kingsborough students are from low- or moderate-income fami- 
lies, and 17-year-olds were admitted to the program with parental consent. 
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borough’s program model as well as the on the key differences between the program and 
the services that were available to the study’s control group. 



Contents of the Report 

Chapter 2 describes how students entered the research sample for the study at 
Kingsborough. It also presents some descriptive characteristics of the sample members and 
describes the data sources used in this report. 

Chapter 3 provides further information about Kingsborough’s Opening Doors pro- 
gram and discusses its implementation. 

Chapter 4 presents the program’s effects on various educational outcomes. 

Chapter 5 provides information on the program’s effects on selected social, psycho- 
logical, and health outcomes. 



16 



Chapter 2 

Sample Intake and Characteristics and Data Sources 



The Opening Doors evaluation is using a random assignment research design to es- 
timate the effects of each college’s program, compared with their regular classes and ser- 
vices. This chapter describes how students became part of the research sample at Kingsbor- 
ough Community College and presents some characteristics of the sample members. It also 
discusses the data sources used in this report and the follow-up periods for the impact 
analyses. 



Identifying, Recruiting, and Randomly Assigning Students 

To enroll students in the study, MDRC and Kingsborough staff worked together to 
insert the recruitment and random assignment procedures into the college’s student registra- 
tion process. This was a considerable challenge, as large community colleges must process 
many student registrations in a short time period. In addition, this was the first time that a 
large-scale random assignment study had been conducted at a community college. The de- 
sign and research procedures were based on past studies set in other contexts, but they had 
to be adapted so that they would disrupt the normal college processes as little as possible. 
The Kingsborough administrators’ and staff members’ commitment to the study, coupled 
with their creativity and flexibility, were critical to the success of the research sample re- 
cruitment and intake processes. 

Potential study participants were identified during the weeks prior to the start of 
each semester. Kingsborough staff began by reviewing lists of applicants who had already 
taken the City University of New York (CUNY) skills assessment tests; as noted in Chapter 
1, scores on the reading and writing tests that are administered before enrollment determine 
most students’ English placements. Applicants whose scores placed them in a developmen- 
tal English course for native English speakers or in freshman English were invited to come 
to campus to register early for classes. 1 

Students who came in to register received a brief, general description of the Open- 
ing Doors program at Kingsborough and were told that the program had sufficient funding 



'As discussed in Chapter 1, because some students were eligible for the English as a Second Language 
(ESL) learning communities program, those whose scores placed them in ESL were not included in Kingsbor- 
ough’s Opening Doors study. 
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to serve about half of eligible freshmen. Further, they were told that the program was part of 
a study, that it was open only to students who agreed to be in the study, and that a random 
process would be used to determine which study participants would be placed in the pro- 
gram. Students who agreed to participate in the study signed an informed consent form, 
provided some baseline demographic information, and completed a brief confidential sur- 
vey. They received a $20 transit card as an incentive and as compensation for their time, 
and they were then randomly assigned either to the program group or to the control group 
and were given appropriate assistance registering for classes. 2 

This sequence was ideal, but, in reality, most Kingsborough freshmen apply and 
take the CUNY skills assessment tests so close to the start of the semester that they were 
unable to attend an early -registration appointment. As a result, the majority of sample 
members entered the study during four or five large registration sessions that occurred in 
the few weeks before each semester began. Opening Doors Learning Communities and 
MDRC staff attended these sessions and “intercepted” freshmen who had just learned their 
test scores and were about to register for classes. Potential study participants heard the ex- 
planation about the research and the learning communities program and, if interested, com- 
pleted the research paperwork in small groups, rather than individually. Random assign- 
ment was conducted on the spot (typically through a phone call to MDRC’s office), and 
students proceeded to register for classes. Chapter 3 describes how registration occurred for 
program group and control group members. 

Students were brought into the research sample in four different groups, or cohorts, 
just before four different semesters: fall 2003, spring 2004, fall 2004, and spring 2005. 
Throughout the study, a total of 1,534 students were randomly assigned at Kingsborough. 



Characteristics of the Sample 

Table 2.1 presents some characteristics of the sample members at Kingsborough, 
based on the questionnaire that they completed just before random assignment: the Baseline 
Information Form (BIF). As the table shows, just over half of the sample members are fe- 
male. The research sample, like the population of Brooklyn, is racially and ethnically di- 



"It is worth noting that students who came to a random assignment appointment and who were placed in 
the control group were — like the program group students — allowed to register for classes earlier than most 
Kingsborough freshmen, and they received advice on the registration process from Opening Doors staff. These 
slightly enhanced services mean that the research design is not completely “pure,” but it was deemed unethical 
and impractical to bring students to campus and then not allow them to register for classes. 
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The Opening Doors Demonstration 
Table 2.1 

Selected Characteristics of Sample Members at Baseline 



Kingsborough Community College Report 



Characteristic 


Percentage 
of Full Sample 


Gender 


Male 


45.4 


Female 


54.6 


Age 


17-18 years old 


44.5 


19-20 years old 


34.2 


21-34 years old 


21.3 


Marital status 


Married 


3.9 


Unmarried 


96.1 


Race/ethnicity a 


Flispanic/Latino 


20.4 


Black, non-Flispanic 


37.7 


White, non-Flispanic 


26.9 


Asian or Pacific Islander 


8.6 


Other 


6.4 


One child or more in household 


8.7 


Flousehold receiving any government benefits 11 


28.4 


Financially dependent on parents 


74.2 


Ever employed 


78.2 


Currently employed 


35.5 


Diplomas/degrees earned 


High school diploma 


70.9 


General Educational Development (GED) certificate 


28.6 


Occupational/technical certificate 


2.0 


Date of high school graduation/GED receipt 


During the past year 


70.2 


Between 1 and 5 years ago 


22.8 


More than 5 years ago 


7.0 


Main reason for enrolling in college 


To complete a certificate program 


2.8 


To obtain an associate's degree 


29.7 


To transfer to a 4-year college/university 


50.2 


To obtain/update job skills 


10.8 


Other 


8.4 



(continued) 
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Table 2.1 (continued) 



Characteristic 


Percentage 
of Full Sample 


First person in family to attend college 


33.4 


Working personal computer in home 


79.7 


Owns or has access to a working car 


25.6 


Language other than English spoken regularly in home 


46.9 


U.S. citizen 


72.6 


Respondent born outside U.S. C 


39.9 


Respondent or respondent's parent(s) bom outside U.S. C 


74.4 


Region in which respondent was bom 


North America 


60.0 


Asia 


6.3 


Commonwealth of Independent States'* 


9.5 


Latin America and the Caribbean 


18.7 


Other 6 


5.5 


f 

Region in which respondent's mother was bom 


North America 


28.2 


Asia 


9.8 


Commonwealth of Independent States 


11.0 


Latin America and the Caribbean 


41.5 


Other 6 


9.6 


Sample size 


1,534 



SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 



NOTES: a Respondents who indicated that they are Hispanic and who also chose a race are included 
only in the Hispanic/Fatino category. 

b Benefits include unemployment/dislocated worker benefits, Supplemental Security Income (SSI) 
or disability, cash assistance or welfare, food stamps, and Section 8 or public housing. 

C "U.S." includes Puerto Rico. 

d This commonwealth comprises Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, 
Republic of Moldova, Russia, Tajikistan, Turkmenistan (until August 2005), Ukraine, and Uzbekistan. 

e Other regions include the Baltics, eastern and western Europe, North Africa, Sub-Saharan Africa, 
the Near East, and Oceania. Countries are grouped by region according to the U.S. Bureau of the 
Census, International Data Base. 

f The majority of respondents reported that both parents were born in the same region as each 

other. 
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verse: 38 percent of the sample members identified themselves as black (non-Hispanic); 27 
percent, as white (non-Hispanic); and 20 percent, as Hispanic. 

Reflecting the makeup of the college’s entering full-time freshmen, the sample 
members were quite young when they entered the study: 45 percent were either 17 or 18, 
and only 21 percent were 21 or older. Very few of the Kingsborough sample members were 
married or had children. Almost three-fourths reported being financially dependent on their 
parents, and roughly one-third were working when they entered the study. 

Most of the sample members (70 percent) had received their high school diploma or 
General Educational Development (GED) certificate during the past year. A small minority 
(7 percent) received their diploma or GED certificate more than five years before entering 
the study. Almost four-fifths of the sample members reported that their main reason for en- 
rolling in college was either to obtain an associate’s degree or to transfer to a four-year in- 
stitution. One-third of the students in the study said that they were the first in their family to 
attend college. 

Almost half of the sample members reported speaking a language other than Eng- 
lish at home — the same proportion as in Brooklyn overall. A full 40 percent of the students 
in the study were born outside the United States: 19 percent were born in Latin America or 
the Caribbean; 1 0 percent were bom in what is now known as the Commonwealth of Inde- 
pendent States (a group of former republics in the Soviet Union); and 6 percent were bom 
in Asia. Almost three-fourths of the sample members reported that either they or at least one 
of their parents were born outside the United States. 

Appendix Table A.l shows the complete list of characteristics that were collected 
on the BIF. The table shows the characteristics for the full sample, the program group, and 
the control group. An asterisk in the rightmost column of the table indicates that the propor- 
tion of program group members with that characteristic is significantly different than the 
proportion of control group members. As the table shows, there are some small differences 
between the two research groups, but no more than would be expected randomly. 3 

As noted above, students at Kingsborough are required to take CUNY skills as- 
sessment tests prior to beginning classes. As Table 2.2 shows, three-fourths of the study’s 
sample members passed the reading test, but only 29 percent passed the writing test, and 29 



Appendix Table A.l does not include an additional statistical test (an omnibus F-test, applied to evaluate 
the joint significance of the individual characteristics); it showed that there are no systematic differences be- 
tween the two research groups. 
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The Opening Doors Demonstration 
Table 2.2 

Outcomes on English Skills Assessment Tests at Baseline 
Kingsborough Community College Report 



Outcome 


Percentage 
of Full Sample 


Passed the reading test 


74.6 


Passed the writing test 


29.3 


Passed both the reading and the writing test 


29.0 


Sample size 


1,534 



SOURCE: MDRC calculations from City University of New York skills assessment test data. 



percent passed both tests. Thus, only a minority of the sample members could avoid devel- 
opmental-level English. 



Data Sources and Follow-Up Periods 

To examine the impacts and implementation of Kingsborough’ s Opening Doors 
program, the analyses presented in this report rely on several data sources, described below. 

Baseline Data 

As mentioned above, just before students were randomly assigned to the study 
groups, they completed a questionnaire, called the Baseline Information Form (BIF), and a 
baseline survey. The BIF collected demographic and other background information. The 
survey asked a series of questions about students’ social and psychological well-being and 
their health. Baseline data were used to describe the research sample, make statistical ad- 
justments in the impact analysis, and define subgroups of sample members for analysis. 

Kingsborough Transcript Data 

Kingsborough provided to MDRC transcript data for the sample members in the 
study. These data include various academic outcomes, including courses registered for, 
withdrawn from, and passed; number of credits earned; and grade point average (GPA). In 
this report, transcript outcomes are presented for the first semester that each sample member 
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was in the study (called the “program semester”) and for the subsequent three semesters 
(called “postprogram semesters”), yielding a follow-up period of two years. Transcript data 
are used in Chapter 4 to provide a detailed look at sample members’ performance at Kings- 
borough. 

CUNY Skills Assessment Test Score Data 

As discussed above, students are required to take the CUNY reading, writing, and 
math skills assessment tests before they begin classes at Kingsborough. MDRC collected 
test score data for all sample members who took the tests at Kingsborough or any other in- 
stitution in the CUNY system. When the analyses for this report were conducted, test score 
data were available through the second postprogram semester. Test score data are used in 
this chapter and in Chapter 4. 

National Student Clearinghouse Data 

The National Student Clearinghouse, a nonprofit organization, collects and distrib- 
utes enrollment, degree, and certificate data from more than 3,000 colleges that enroll more 
than 90 percent of the nation’s college students. 4 The Clearinghouse data are used in Chap- 
ter 4 to provide important information about students in the study who may have attended a 
postsecondary institution other than Kingsborough. The Clearinghouse data provide infor- 
mation through the third postprogram semester, covering two years after sample members 
were randomly assigned. 

The Opening Doors 12-Month Survey 

A survey was administered to sample members approximately 12 months after ran- 
dom assignment. 5 MDRC attempted to locate and interview all sample members; in the end, 
7 1 percent of the full research sample at Kingsborough completed the survey. The survey 
asked about a wide range of topics, including sample members’ educational experiences, 
social relationships and supports, future outlook and identity, and health. Selected measures 
from the survey are used in Chapters 4 and 5. 



4 National Student Clearinghouse (2007). 

5 The Opening Doors 12-Month Survey used some questions from the Community College Survey of Stu- 
dent Engagement, with permission. 
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The Kingsborough Student Survey 

During fall 2004, MDRC surveyed program and control group members who had 
entered the study that semester. 6 The survey measured students’ experiences at Kingsbor- 
ough, including activities and experiences in their courses, relationships with other students, 
and awareness of student services. The survey was targeted to all 242 program group mem- 
bers enrolled in an Opening Doors learning community that semester and to a random sub- 
set of 138 control group members; 84 percent of the program group members and 73 per- 
cent of the control group members responded. Because the sample size is much smaller than 
the sample for the Opening Doors 12-Month Survey, and because the Kingsborough Stu- 
dent Survey is limited to only one of the four research cohorts, the data are used to illustrate 
some important information about program implementation in Chapter 3, but the report’s 
impact analyses do not rely on the Kingsborough survey data. 

The Kingsborough Faculty Survey 

In early 2005, MDRC distributed a survey to the faculty members who taught in the 
Opening Doors Learning Communities program during the fall 2004 semester. The survey 
asked a series of questions about the faculty members’ experiences teaching in the learning 
community and covered such topics as interactions with students and other faculty members 
and the curricula and instruction in the learning community courses. Of the 24 faculty 
teaching in an Opening Doors Learning Community that semester, 21 completed the survey. 
Information from the faculty survey is used in Chapter 3 to help describe the implementa- 
tion of the Opening Doors program at Kingsborough. 

Field Research 

During spring 2005, MDRC staff conducted a series of field research visits to 
Kingsborough. MDRC interviewed many college administrators, faculty, and staff, includ- 
ing those involved in Opening Doors. The interviews provided detailed information on the 
operation of the Opening Doors Learning Communities program and about the key differ- 
ences between the program and the standard college courses and services that were avail- 
able to the members of the study’s control group. Throughout the study, prior to the formal 
field research visits, MDRC periodically interviewed the administrators and staff involved 
in Opening Doors and observed meetings between counselors and students. Information 
from all these visits is used in Chapter 3 . 



6 The Kingsborough Student Survey was adapted from the Pathways to Success Survey, created by Vin- 
cent Tinto. The Pathways to Success Survey is a modification of the Community College Survey of Student 
Engagement and was used with permission. 
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The Opening Doors Qualitative Study 

MDRC conducted an in-depth qualitative study at Kingsborough and another Open- 
ing Doors college, Lorain County Community College in Elyria, Ohio, outside Cleveland. 
During spring 2005, interviews were conducted with 23 students from the program and con- 
trol groups at Kingsborough to learn about their experiences in college and the factors that 
affected their ability to stay in school. The study’s results were presented in a report in 
2006, 7 and some of the interview data are used in Chapter 3. 



7 Gardenhire-Crooks, Collado, and Ray (2006). 
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Chapter 3 



The Implementation of 
Opening Doors Learning Communities 



The chapter describes how the Opening Doors Learning Communities program operated 
at Kingsborough Community College during the study period. After a brief summary, the chapter 
describes the program model and structure, examines how Kingsborough administrators mounted 
the program in fall 2003, discusses key operational issues that emerged over time, and finally de- 
scribes how Opening Doors Learning Communities differed from the regular college environment 
facing the study’s control group. 

The key findings are: 

• Opening Doors Learning Communities was well implemented at Kingsbor- 
ough. Despite a compressed planning period and the program’s large scale, 
all the key features were in place when operations began, and they remained 
in place throughout the study period. In interviews, many faculty, students, 
and administrators expressed positive views about the program. These ac- 
complishments were possible because the program received strong, consis- 
tent support from the highest levels of the college administration. 

• All the learning communities had the same basic structure, but they varied in 
their content and class sizes and in the degree to which faculty worked to- 
gether and integrated their courses. Thus, while this study is a strong test of 
the structural features of a learning community (linked courses, block sched- 
uling, and so on), it may not fully test the impacts of tightly integrating 
course curricula. That said, it seems likely that Kingsborough’ s program is at 
least as strong as, if not stronger than, the “typical” learning communities 
program at a community college, and so the results are probably widely ap- 
plicable. 

Finally, it is important to note that Kingsborough is, in general, a high-functioning com- 
munity college. Thus, the study’s control group probably received relatively high-quality instruc- 
tion and relatively strong support. This means that the benchmark for achieving impacts may be 
higher at Kingsborough than it might have been at another college. 
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The Opening Doors Learning Communities Model 

The Opening Doors Learning Communities program began in fall 2003 and still operates 
today. However, members of the Opening Doors research sample experienced the program only 
during its first four semesters of operation, from fall 2003 through spring 2005 (known as the 
“study period”)- After the study period ended, Kingsborough expanded the program, with the goal 
of serving 80 percent of entering freshmen by 2010, and it made some modest changes in the 
model. At the time this report was written, about 65 percent of entering freshmen participated in 
Opening Doors Learning Communities. This chapter reflects the model and operations only dur- 
ing the study period (and thus is written in the past tense). 

Program Components 

Opening Doors Learning Communities used the common paired- or clustered-course 
model described in Chapter 1 . That is, the students in each learning community took three courses 
together: an English course, with the level determined by the student’s scores on the City Univer- 
sity of New York (CUNY) reading and writing skills assessment tests that were administered prior 
to enrollment; an academic course required for their major, called a “content course” (for exam- 
ple, psychology, health, or history); and a one-credit freshman orientation class. This class, which 
is open to all Kingsborough freshmen, teaches tune management, study skills, college rules and 
procedures, exploration of learning styles, career exploration, multicultural diversity, and other 
topics relevant to a new college student; similar courses are offered at many two-year and four- 
year colleges. As discussed later in the chapter, 31 of the 40 learning communities that operated 
during the study period included a developmental English course; the other 9 included a credit- 
bearing freshman English course for students who passed both the reading and the writing skills 
assessment tests prior to enrollment. 

Figure 3.1 shows the actual class schedules for 2 of the 1 1 Opening Doors learning com- 
munities that operated during fall 2004, the third semester of the study period. The first panel of 
the figure shows the schedule for a learning community that linked a credit-bearing (that is, non- 
developmental) English course (English 12) with a sociology course (Sociology 31) and the 
freshman orientation course (Student Development [SD] 10). The second panel shows the sched- 
ule for a learning community that linked the lowest-level developmental English course (English 
91) with a health course that is required for most Kingsborough students (Health 12) and the 
freshman orientation class. 
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The Opening Doors Demonstration 
Figure 3.1 

Schedules Showing Linked Courses in the 
2004 Opening Doors Learning Communities Program 

Kingsborough Community College Report 



Sociology Link (10) 

English 12 (ENG 12) and Sociology 31 (SOC 31) 



Period 


Monday 


Tuesday 


Wednesday 


Thursday 


Friday 


A 

(8:00-9:00) 












B 

(9:10-10:10) 












C 

(10:20-11:20) 












D 

(11:30-12:30) 


ENG 000LB 

(0.0 Credits) 










E 

(12:40-1:40) 


SOC 31 

(3.0 Credits) 


SD 10 

(1.0 Credit) 


SOC 31 
(3.0 Credits) 


SOC 31 
(3.0 Credits) 




F 

(1:50-2:50) 


ENG 12 
(4.0 Credits) 


ENG 12 
(4.0 Credits) 


ENG 12 
(4.0 Credits) 


ENG 12 

(4.0 Credits) 




G 

(3:00-4:40) 













(continued) 



As Figure 3.1 shows, the Opening Doors courses met one after the other (sometimes with 
a break for lunch). This scheduling system was designed to min imize the amount of time that a 
student needed to be on campus, making it easier to balance school with work and family obliga- 
tions. In addition, all Opening Doors courses met at convenient times, generally between 9 A.M. 
and 3 P.M., Monday through Thursday. 1 



'After the study period for this report ended, Kingsborough began offering learning communities for eve- 
ning students as well. 
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Figure 3.1 (continued) 



Health Link (1) 

English 91 (ENG 91) and Health 12 (HPE 12) 



Period 


Monday 


Tuesday 


Wednesday 


Thursday 


Friday 


A 

(8:00-9:00) 












B 

(9:10-10:10) 












C 

(10:20-11:20) 


ENG 91 

(8.0 EQ Credits) 


ENG 91 
(8.0 EQ Credits) 


ENG 91 
(8.0 EQ Credits) 


ENG 91 
(8.0 EQ Credits) 




D 

(11:30-12:30) 


ENG 91 

(8.0 EQ Credits) 


ENG 91 
(8.0 EQ Credits) 


ENG 91 
(8.0 EQ Credits) 


ENG 91 
(8.0 EQ Credits) 




E 

(12:40-1:40) 












F 

(1:50-2:50) 


HPE 12 

(3.0 Credits) 


HPE 12 

(3.0 Credits) 


HPE 12 

(3.0 Credits) 


SD 10 

(1.0 Credit) 




G 

(3:00-4:30) 













SOURCE: Kingsborough Community College. 



NOTES: The schedules represented here are actual schedules from Kingsborough Community College. 
Linked courses form the core of the Opening Doors Learning Communities program at Kingsborough. 
All combinations of Kingsborough's linked courses include one English course; one "content course" 
(covering an academic subject other than English); and Student Development 10 (SD 10), a skills 
course for new students. 

"EQ credits" stands for equated credits. Equated credits are weekly class hours in developmental 
and compensatory courses for which actual credit is not allowed. For certain purposes, such as 
determining financial aid eligibility, equated credits may be counted in the same manner as regular 
credits. 



A typical full-time course load at Kingsborough involves 12 credits (12 hours of class per 
week). Because the lower-level developmental English courses meet for eight hours each week, 
the content courses are typically three credits, and the freshman orientation class is one credit, stu- 
dents at the lower English levels usually took no additional unlinked courses. In contrast, students 
in higher-level English courses, which meet for fewer hours per week, usually took at least one 
non-Opening Doors course. For example, the linked Opening Doors courses for the schedule 
shown in the first panel of Figure 3. 1 total only eight credits. 
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Faculty who taught the linked courses had a reduced teaching load, allowing them to meet 
regularly during the semester to discuss student progress, identify strategies to assist students hav- 
ing difficulty, and coordinate assignments. In effect, a three-credit course taught in the context of 
Opening Doors was treated as a four-credit course when determining a faculty member’s teaching 
load. (After the first semester, participating faculty also received compensation for time spent 
planning their Opening Doors course during the six-week module preceding the semester.) 

In addition to the linked-course structure, the Opening Doors Learning Communities pro- 
gram included several other components designed to address students’ barriers to retention and 
academic success: 

• Enhanced counseling and support. In each learning community, the fresh- 
man orientation course was taught by an Opening Doors counselor (usually 
called a “case manager”), who worked proactively to identify and resolve 
students’ barriers to good attendance and performance. Ideally, the case man- 
ager/instructor participated in regular meetings with the other two faculty 
members in a given learning community during the semester, creating an ef- 
fective “early-warning” system to identify students needing assistance — for 
example, students who had been missing classes or who were having diffi- 
culty with assignments. Typically, each Opening Doors case manager was 
responsible for three or four learning communities (75 to 100 students in all). 

• Enhanced tutoring. Normally, tutors are assigned to developmental English 
courses at Kingsborough — and may actually attend the classes — but oth- 
erwise students access tutoring by visiting a central lab. In the Opening 
Doors Learning Communities program, a tutor was assigned to each learning 
community and attended the English course and, in many cases, the subject- 
matter course as well. The aim was to ensure that tutors were familiar with 
the material being covered — and with the students. They were well posi- 
tioned both to help with the work in a given course and to help students draw 
connections across the linked courses. 

• Textbook vouchers. The high cost of college textbooks has been well 
documented, and studies have shown that many community college students 
do not purchase the books for their courses; they attempt to share or borrow 
books or simply get by without them. 2 Redeemable at the campus bookstore, 
the Opening Doors textbook voucher was worth up to $150 during the initial 



2 Community College Week (2003). 
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